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FOREWORD 


The  Army  Career  Transition  Survey  (ACTS)  was  an  exit  survey  administered  to  soldiers 
leaving  the  Army.  The  ACTS  was  designed  to  measure  soldiers’  satisfaction  with  various 
aspects  of  Army  life  and  to  determine  whether  dissatisfaction  with  these  aspects  was  related  to 
their  leaving  the  Army. 

The  proponent  for  the  ACTS  and  for  similar  items  in  the  Sample  Survey  of  Military 
Personnel  (SSMP)  was  the  Headquarters,  Department  of  Army  Deputy  Chief  of  Staff  for 
Personnel  (HQDA).  Other  Army  agencies  with  a  need  for  the  results  were  the  Total  Army 
Personnel  Command,  U.S.  Army  Recruiting  Command,  and  the  U.S.  Army  Community  and 
Family  Support  Center. 

The  purpose  of  the  study  described  in  this  report  was  to  analyze  the  current  ACTS  data, 
to  compare  responses  to  ACTS  items  with  responses  to  comparable  SSMP  items,  and  to  make 
recommendations  about  continuing  administration  of  the  ACTS. 

The  Army  can  use  the  findings  contained  in  this  report  to  make  decisions  about  the  most 
practical  means  for  collecting  data  on  soldier  satisfaction  and  reasons  for  leaving  the  Army. 


Director 


* 


V 


Preceding  Page!  Blank 


ANALYSIS  OF  THE  REVISED  ARMY  CAREER  TRANSITIONS  SURVEY  (ACTS)  AND 
COMPARISON  WITH  THE  FALL  1996  SAMPLE  SURVEY  OF 

MILITARY  PERSONNEL  (SSMP):  RESULTS  AND  RECOMMENDATIONS 

EXECUTIVE  SUMMARY _ _ _ 

Research  Requirement: 

From  1990-1993,  the  U.S.  Army  Research  Institute  (ARI)  administered  a  pilot  exit  survey 
to  separating  soldiers  known  as  the  Army  Career  Transitions  Survey  (ACTS).  The  ACTS  is  an 
exit  survey  of  soldiers  who  are  leaving  the  Active  Component  Army.  The  survey  was  designed 
to:  a)  measure  soldiers’  satisfaction  with  various  aspects  of  Army  life  and  b)  determine  whether 
their  dissatisfaction  with  an  aspect  of  Army  life  was  an  important  reason  for  their  leaving  the 
Army.  Tracking  trends  in  the  survey  results  is  thought  to  provide  the  Army  with  indications  of 
key  issues  related  to  retention  of  soldiers.  The  proponent  for  the  ACTS  was  the  Headquarters, 
Department  of  Army  (HQDA)  Deputy  Chief  of  Staff  for  Personnel.  Other  Army  agencies  with  a 
need  for  the  results  were  the  Total  Army  Personnel  Command,  U.S.  Army  Recruiting  Command, 
and  the  U.S.  Army  Community  and  Family  Support  Center  (CFSC).  This  pilot  version  of  the 
ACTS  was  never  accorded  full  status  as  the  Army’s  exit  survey  instrument  and  was  suspended 
pending  further  inquiry  on  the  instrument. 

In  1995,  the  Human  Resources  Research  Organization  (HumRRO)  was  awarded  a 
contract  to  develop  standardized  administration  procedures  and  to  review  and  revise  the  items  in 
the  ACTS.  The  results  of  that  contract  determined  that  several  problems  did  exist.  HumRRO 
identified  these  problems,  offered  recommendations  for  solving  the  problems,  and  updated  the 
ACTS. 


The  objectives  of  the  present  contract  were  to  (a)  review  the  results  of  the  revised  version, 

(b)  compare  ACTS  responses  to  those  of  the  Sample  Survey  of  Military  Personnel  (SSMP),  and 

(c)  provide  recommendations  regarding  the  future  of  the  ACTS.  Because  many  of  the  questions 
contained  in  the  ACTS  were  also  in  the  semi-annual  SSMP,  the  results  of  the  two  surveys  might 
be  similar.  If  this  is  the  case,  then  administration  of  the  ACTS  may  not  be  necessary.  The 
principal  goal  of  this  contract  was  to  determine  this  necessity. 

Procedure: 

To  accomplish  the  objectives  of  this  contract,  the  approach  involved  an  analysis  of  the 
current  ACTS  data  followed  by  a  comparative  analysis  of  the  ACTS  and  SSMP  responses  for 
identical  questions.  Recommendations  would  be  based  on  these  two  analyses. 

Findings: 

For  the  first  objective,  ACTS  items  were  analyzed  in  accordance  with  the  requests  of  ARI 
and  followed  ARI  instructions  regarding  appropriate  statistical  methodology  and  variables  of 
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interest.  Analyses  revealed  that  among  items  receiving  the  highest  percentage  of  satisfaction 
ratings  were:  Overall  Assistance  Provided  by  the  Army  Career  and  Alumni  Program  (ACAP) 
(93%),  Army  Community  Service  (ACS)  Programs  (92.4%),  Explanation  of  Transition  Benefits 
and  Entitlements  (87.1%),  and  Army  Youth  Services  Programs  (80.4%).  The  items  receiving  the 
lowest  percentage  of  satisfaction  ratings  were:  Amount  of  Time  for  Family  and  Friends  (46.9%), 
Amount  of  Personnel  to  Do  Work  (45.7%),  Respect  Army  Shows  Soldiers  (42.3%),  Respect 
Army  Shows  Single  Soldiers  (41 .7%),  Army  Support  and  Concern  for  Family  (41%),  and 
Amount  of  Time  Separated  from  Family  (35%). 

Differences  in  the  satisfaction  level  reported  within  demographic  categories  revealed  that 
the  greatest  number  of  differences  occurred  for  rank  groups  (officers/enlisted),  race 
(black/white),  and  marital  status  (married/single).  Generally,  officers  were  more  satisfied  than 
enlisted  personnel  across  categories,  with  Sergeants-Sergeant  Majors/Command  Sergeant  Majors 
reporting  more  satisfaction  than  the  lower  ranks.  This  pattern  was  reversed  for  family  medical 
and  dental  care  availability,  PCS  compensation  moves,  retirement  benefits  and  amount  of  time 
separated  from  family,  with  higher  ranks  reporting  less  satisfaction  than  lower  ranks.  Differences 
between  blacks  and  whites  tended  to  show  that  blacks  were  more  satisfied  than  whites,  although 
this  pattern  was  reversed  for  job  security,  overseas  duty,  overall  preparation  for  getting  a  civilian 
job,  fairness  of  performance  evaluations,  and  special  pay.  Married  personnel  were  more 
satisfied  than  single  personnel  across  categories,  although  having  children  did  not  greatly  affect 
the  satisfaction  of  single  or  married  personnel  on  most  issues. 

The  ACTS  responses  revealed  a  clear  pattern  in  “reason  for  leaving”  ratings.  Among  the 
highest  percentage  items  that  ACTS  respondents  perceived  as  major  reasons  for  leaving  were: 
Amount  of  Time  Separated  from  Family  (30.2%),  Respect  Army  Shows  Soldiers  (29.4%), 
Promotion/Advancement  Opportunity  (25.3%),  Quality  of  Senior  NCO  Leadership  (24.4%), 
Overall  Quality  of  Army  Life  (23.6%),  Amount  of  Time  for  Family  and  Friends  (22.4%), 

Respect  from  Superiors  (22.3%),  Respect  Army  Shows  Single  Soldiers  (22.3%),  Army  Support 
and  Concern  for  Family  (22.2%),  Recognition  for  Accomplishments  (21.8%),  Supervisors 
Competence  (21 .7%),  Quality  of  Officer  Leadership  (21 .2%),  and  Control  over  Job  Assignments 
(20.1%). 

Among  the  lowest  percentage  items  that  ACTS  respondents  perceived  as  major  reasons 
for  leaving  were:  Army  Community  Service  Programs  (2.8%),  Army  Youth  Service  Programs 
(2.9%),  Overall  Assistance  Provided  by  the  Army  Career  and  Alumni  Program  (3.2%), 
Explanation  of  Transition  Benefits  and  Entitlements  (4.1%),  Morale,  Welfare,  and  Recreation 
(MWR)  Programs  (4.1%),  and  DoD  Dependent  Schools  (DODDS)  (4.4%). 

The  demographic  differences  in  “reason  for  leaving”  percentages  were  comparatively 
few.  In  general,  these  differences  showed  that  females  and  blacks  were  more  likely  than  males 
and  whites  to  believe  that  a  particular  aspect  of  Army  life  was  a  more  significant  reason  for 
leaving.  Similarly,  enlisted  personnel  were  more  likely  to  believe  that  particular  aspects  of  Army 
life  were  significant  reasons  for  leaving  than  were  officers,  although  these  differences  were  more 
common  for  Privates-Specialist/Corporals  than  for  Sergeants-Sergeants  Majors/Command 
Sergeant  Major. 
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Marital  status  was  frequently  an  important  factor  in  determining  whether  an  aspect  of 
Army  life  was  a  significant  reason  for  leaving  the  Army.  Generally,  larger  percentages  of 
married  than  single  personnel  believed  that  particular  aspects  of  Army  life  were  reasons  to  leave 
the  Army.  Involuntary  separations  and  those  who  were  not  seeking  a  similar  job  outside  the 
Army  were  also  more  likely  to  find  a  particular  aspect  of  Army  life  to  be  a  major  reason  for 
leaving  the  Army  than  voluntary  separations  and  those  who  were  seeking  a  similar  job.  Having 
children  and  one’s  educational  level  were  unrelated  to  “reason  for  leaving”  ratings. 

One  pattern  found  in  ACTS  responses  was  an  inverse  relationship  between  “satisfaction” 
and  “reasons  for  leaving” — that  is,  items  with  the  highest  levels  of  satisfaction  were  items  with 
the  lowest  percentage  of  ratings  as  major  reasons  for  leaving,  while  all  but  one  of  the  items  that 
had  the  lowest  satisfaction  ratings  were  items  with  the  highest  percentage  of  personnel  who 
perceived  them  as  major  reasons  for  leaving.  Army  retention  may  be  adversely  affected  by  this 
pattern. 


For  the  second  objective  (the  primary  part  of  the  study),  a  log-linear  analysis  was 
performed  to  determine  whether  the  responses  to  the  SSMP  and  ACTS  surveys  were  identical 
overall,  as  well  as  to  determine  whether  there  were  any  particular  differences  within 
demographic  category  responses.  Using  ARI’s  standard  for  the  equivalence  items  on  the  two 
surveys,  it  was  found  that  there  was  no  overall  statistical  difference  between  ACTS  and  SSMP 
responses.  Essentially,  on  the  items  that  the  two  surveys  share,  there  were  no  differences  in  the 
responses  provided.  As  a  result,  the  ACTS  provides  information  redundant  to  that  provided  by 
the  SSMP. 

The  third  objective  in  the  contract  required  that  two  types  of  recommendations  be  made 
as  a  result  of  the  data  analyses  done  to  meet  the  objectives  stated  above.  An  overall 
recommendation  was  to  be  made  concerning  the  continuation  of  the  ACTS.  In  addition,  a  second 
group  of  recommendations  was  to  be  made  regarding  the  ACTS  should  the  Army  choose  to 
continue  its  administration.1 

Utilization  of  Findings: 

We  recommend  that  the  Army  discontinue  the  ACTS  because  the  results  from  the 
matching  ACTS  and  SSMP  items  do  not  differ  statistically.  Thus  there  is  no  basis  for  continuing 
the  administration  of  the  ACTS. 


1  The  second  group  of  recommendations,  contained  in  Appendix  B,  was  made  in  accordance  with 
the  contractual  stipulations  and  in  no  way  should  be  construed  as  a  recommendation  to  continue 


the  ACTS. 
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Recommendations 


INTRODUCTION 

Background 

The  Army  Career  Transitions  Survey  (ACTS)  was  an  exit  survey  given  soldiers  leaving 
the  Active  Component  Army.  It  was  designed  to  measure  soldiers’  satisfaction  with  various 
aspects  of  Army  life  and  to  determine  whether  their  dissatisfaction  with  an  aspect  of  Army  life 
was  an  important  reason  for  leaving  the  Army.  As  with  other  exit  survey  data,  tracking  trends  in 
the  results  was  intended  to  provide  the  Army  with  indications  of  key  issues  related  to  soldier 
retention.  From  1990  through  1993,  the  U.S.  Army  Research  Institute  (ARI)  administered  an 
experimental  exit  survey  to  separating  soldiers.  This  instrument,  the  ACTS,  was  designed  for  use 
among  separating  Active  Component  Army  personnel  to  determine  satisfaction  levels, 
perceptions  of  Army  leadership,  advice  for  potential  recruits,  and  the  reasons  for  leaving  the 
Army.  This  pilot  version  of  the  ACTS,  which  was  never  accorded  full  status  as  the  Army’s  exit 
survey  instrument,  was  suspended  pending  further  examination  of  the  instrument.  The  purpose  of 
the  resulting  inquiry  was  to  develop  standardized  administration  procedures  as  well  as  to  review 
and  revise  the  items  on  the  survey.  The  findings  of  this  analysis  revealed  that  several  problems 
did  exist.  These  problems  were  identified,  recommendations  were  made  for  solving  the 
problems,  and  the  ACTS  was  updated.. 

The  proponent  for  the  ACTS  and  for  the  similar  items  in  the  Sample  Survey  of  Military 
Personnel  (SSMP)  was  the  Headquarters,  Department  of  Army  Deputy  Chief  of  Staff  for 
Personnel  (HQDA).  Other  Army  agencies  with  a  need  for  the  results  were  the  Total  Army 
Personnel  Command,  U.S.  Army  Recruiting  Command,  and  the  U.S.  Army  Community  and 
Family  Support  Center. 

ACTS  items.  The  pilot  version  of  the  ACTS  consisted  of  104  items,  each  with  multiple 
response  options.  These  items  can  be  broadly  grouped  into  four  areas:  a)  background 
information  and  demographics;  b)  satisfaction  with  various  aspects  of  the  Army  environment; 
c)  satisfaction  with  Army  leadership  (Army  Leadership  Supplement);  and  d)  questions  related  to 
occupational  training  and  duties.  The  instrument  typically  required  1 5  to  25  minutes  to 
complete.  Giacalone’s  (1993)  evaluation  of  the  ACTS  raised  a  number  of  methodological  and 
administrative  issues,  including  the  lack  of  demonstrable  utility  and  inconsistent  survey 
administration  and  data  tracking  procedures.  Additionally,  problems  with  the  data  (low  survey 
response  rates  and  different  response  patterns  for  unidentified  and  identified  respondents  and  for 
involuntary  and  voluntary  separations)  raised  additional  concerns.  Thus,  Giacalone  offered  four 
primary  recommendations:  a)  determine  whether  the  collected  data  will  address  the  needs  of  a 
range  of  specific,  potential  users;  b)  update  and  revise  the  satisfaction  measures;  c)  standardize 
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the  administration  procedures;  and  d)  measure  the  impact  of  respondent  identification  on  the 
resulting  data. 

Giacalone’s  evaluation  led  to  an  effort  to:  a)  identify  administration  problems  and 
develop  standardized  administration  procedures,  and  b)  review  and  revise  the  items  on  the 
ACTS.  The  results  of  the  findings  are  summarized  below. 

Administration  problems:  Identification  and  recommendations.  Interviews  with 
transition  center  personnel  determined  how  the  ACTS  was  implemented  at  each  location,  what 
problems  were  occurring,  and  what  improvements  should  be  considered.  The  data  were 
analyzed,  and  five  common  problem  areas  were  identified. 

Problem  1:  Many  sites  were  unaware  of  their  own  low  return  rates  and  administration 
problems. 

Problem  2:  Because  the  ACTS  was  voluntary,  it  was  more  likely  to  be  disregarded  by  soldiers 
and  by  those  responsible  for  its  administration. 

Problem  3:  Administrative  support  systems  to  help  direct  and  monitor  ACTS  transition  site 
activities  were  lacking. 

Problem  4:  An  enormous  disparity  existed  between  the  number  of  personnel  separating  across 
the  various  transition  points. 

Problem  5:  Transition  sites  had  slightly  more  difficulty  obtaining  completed  ACTS  from  officers 
than  from  enlisted  personnel. 

A  number  of  recommendations  were  made  to  deal  with  the  problems: 

1 .  Make  the  ACTS  mandatory. 

2.  Require  attendance  at  the  pre-separation  briefings  and  inform  officers  of  the  survey’s 
importance. 

3.  Provide  clearer  directions  to  transition  sites  regarding  how  and  when  the  ACTS  should 
be  administered. 

4.  Encourage  cooperation  and  compliance  by  conducting  group  administration  of  the 
ACTS  prior  to  day  of  separation,  such  as  during  the  pre-separation  and  pre-retirement 
briefings  approximately  3  to  6  months  before  separation. 
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5.  Provide  the  name  and  telephone  number  of  an  ACTS  contact  person  with  every 
shipment.  Supplies  should  be  sent  to  the  sites  on  a  routine  basis. 

6.  Make  certain  that  forms  are  returned  to  ARI. 

7.  Require  transition  points  to  account  for  low  response  rates. 

8.  Establish  quarterly  deadlines  for  the  return  of  the  completed  ACTS. 

9.  Provide  transition  point  personnel  with  quarterly  feedback  regarding  their  response 
rates  and  related  problems. 

1 0.  Provide  more  administration  information  for  transitioning  personnel. 

1 1 .  Provide  support  for  overburdened  transition  points  to  handle  large  groups  of 
separating  personnel,  taking  monthly  fluctuations  into  account. 

Review  and  revision  of  ACTS  items.  Giacalone  (1993)  noted  several  deficiencies  in  the 
ACTS  survey  items  including:  a)  the  lack  of  evidence  to  suggest  that  important  Army  issues  had 
been  addressed  by  the  survey;  b)  unclear  items  subject  to  multiple  interpretations;  and  c)  lack  of 
any  known  Army  data  needs  that  were  addressed  by  the  survey  items.  As  a  result  of  Giacalone’ s 
findings,  interviews  with  major  Army  commands  were  conducted  to  identify  the  needs  of  the 
sponsors  and  to  determine  how  well  the  ACTS  had  met  these  needs. 

In  accordance  with  directives  from  ARI  and  from  developed  leads,  the  Army  Career  and 
Alumni  Program,  Army  Housing  Office,  Army  Community  and  Family  Support  Center,  U.S. 
Army  Recruiting  Command,  and  Department  of  the  Army,  Personnel  (DAPE)  were  interviewed 
and/or  surveyed  and  ultimately  initiated  the  revision  of  ACTS  items  and/or  the  analyses. 

Three  significant  changes  were  made  to  the  satisfaction  items.  First,  13  items  that  failed 
to  load  on  any  factor  during  a  previous  factor  analysis  (see  Giacalone,  Elig,  Ginexi,  &  Bright, 

1 995),  and  that  were  not  of  interest  to  any  command,  were  deleted.  Second,  an  “importance” 
rating  was  added  to  the  satisfaction  rating  scale.  This  new  rating  procedure  was  designed  to 
indicate  how  critical  satisfaction  items  (and  different  aspects  of  the  Army)  are  viewed  by  the 
soldiers  and  was  expected  to  increase  data  quality.  It  was  recommended  that  an  index  be  devised 
to  weight  the  item  “importance”  of  each  satisfaction  rating.  Finally,  the  last  change  involved  an 
item  in  the  original  ACTS  which  asked  the  respondent  to  indicate  the  most  important  reason  why 
s/he  left  the  Army  (item  C).  This  item  was  excluded  in  the  revised  survey  due  to  lack  of  interest . 

Pilot  administration  of  the  revised  ACTS.  Transition  site  personnel  were  instructed  to 
administer  the  revised  version  to  soldiers  on-site;  480  surveys  were  returned.  Analysis  revealed 
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eight  distinct  factors:  leadership/supervisory,  benefits/support  services,  training,  family  issues, 
personal  benefits,  family  benefits,  PERSTEMPO,  and  overall  quality  of  life. 

Objectives  of  the  Current  Investigation 

Based  on  the  assumption  that  the  results  would  be  provided  to  senior  Army  leadership  for 
use  in  developing  plans,  assessing  policies,  and  evaluating  program  operations  and  outcomes,  the 
current  investigation  sought  to  analyze  the  ACTS  and  similar  SSMP  question  data  in  order  to 
achieve  three  objectives. 

Objective  1:  Determine  results  of  the  ACTS  to  date,  indicating  why  soldiers  are 
leaving  the  Army  (including  differences  between  those  with  different  personal  and  military 
demographics)  and  their  level  of  satisfaction/dissatisfaction  with  various  aspects  of  Army 
life.  The  analyses  sought  to  answer  the  following  questions: 

1 .  What  indices  can  be  developed  to  create  more  stable  measures  of 
soldiers’  concerns  about  Army  life  and  reasons  for  leaving  the  Army? 

2.  What  is  (are)  the  primary  [most  frequently  given]  reason(s)  for  leaving  the 
Army? 

3.  How  strongly  are  levels  of  satisfaction/dissatisfaction  related  to  the  reasons 
indicated  for  leaving  the  Army? 

Where  sufficient  numbers  of  respondents  were  available  in  each  analysis  cell,  comparison 
among  subgroups  of  soldiers  were  to  be  made  in  the  following  categories:  Officers  vs.  enlisted 
(Q3),  Pay  grade  (Q3-4),  Rank  subgroups  (Q3-4),  Branch  or  Military  Occupational  Specialty 
(MOS)(Q17),  Plan  to  look  for  job  similar  to  Branch  or  MOS  (Q 12),  Marital  status  (Q6),  Race, 
ethnic  origin  (Q9- 10),  Gender  (Q7),  Transition  status  (Q1 1),  Education  level  (Q 16),  and 
Dependent  family  members  (Q6,  Q8). 

Objective  2:  Compare  the  “satisfaction/dissatisfaction  with  various  aspects  of  Army 
life”  in  ACTS  results  with  results  for  similar  items  from  the  Fall  1996  SSMP.  The  purpose 
of  comparing  ACTS  and  SSMP  results  was  to  determine  whether  existing  survey  data  (collected 
semi-annually  by  the  SSMP)  provided  Army  leaders  with  the  same  information  as  collected  by 
ACTS.  This  objective  was  considered  the  primary  objective  of  the  overall  study. 

If  the  same  information  is  available  from  both  ACTS  and  SSMP,  there  would  be  no  need 
for  the  ACTS.  Since  many  of  the  items  are  not  the  same  on  the  two  questionnaires  (although  a 
substantial  number  of  items  are  redundant),  not  all  of  the  results  of  the  two  surveys  are  reported 
here. 

Objective  3:  Make  recommendations  about  the  continuation  of  ACTS.  The  results 
of  the  ACTS  analyses  would  be  provided  to  senior  Army  leadership  for  use  in  developing 
plans,  assessing  policies,  and  evaluating  program  operations  and  outcomes. 
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Given  these  three  objectives,  we  now  turn  to  the  results  of  the  investigation  and  the 
details  of  the  findings  for  each  of  the  objectives. 

FINDINGS 

Objective  1 :  Determine  ACTS  Results  to  Date 

ACTS  satisfaction  items:  Highest  and  lowest  satisfaction  percentages.  The  ACTS 
responses  revealed  a  clear  pattern  of  satisfaction.  Among  the  highest  percentages  of  satisfaction 
were  the  items:  Overall  Assistance  Provided  by  the  Army  Career  and  Alumni  Program  (ACAP) 
(93%),  Army  Community  Service  (ACS)  Programs  (92.4%),  Explanation  of  Transition  Benefits 
and  Entitlements  (87.1%),  and  Army  Youth  Services  Programs  (80.4%). 

The  lowest  percentages  of  satisfaction  were  the  items:  Amount  of  Time  Separated  from 
Family  (35%),  Army  Support  and  Concern  for  Family  (41%),  Respect  Army  Shows  Single 
Soldiers  (41 .7%),  Respect  Army  Shows  Soldiers  (42.3%),  Amount  of  Personnel  to  Do  Work 
(45.7%),  and  Amount  of  Time  for  Family  and  Friends  (46.9%). 

ACTS  satisfaction  items:  Important  demographic  differences  in  satisfaction  percentages. 
Although  there  were  numerous  differences  in  the  satisfaction  level  reported  within  demographic 
categories,  the  greatest  number  of  differences  involved  rank  groups  (officers/enlisted),  race 
(black/white),  and  marital  status  (married/single). 

In  general,  officers  were  more  satisfied  than  enlisted  personnel  across  categories;  a 
similar,  although  a  less  pronounced,  pattern  is  evident  in  comparing  Privates  through 
Specialists/Corporals  to  Sergeants  through  Sergeant  Majors/Command  Sergeant  Majors,  with 
higher  ranks  reporting  more  satisfaction.  On  the  issues  of  family  medical  and  dental  care 
availability,  PCS  compensation  moves,  retirement  benefits,  and  “amount  of  time  separated  from 
family,”  these  patterns  were  reversed,  with  higher  ranks  reporting  less  satisfaction.  Differences 
between  blacks  and  whites  tended  to  show  that  blacks  were  generally  more  satisfied  than  whites, 
although  this  pattern  was  reversed  for  job  security,  overseas  duty,  “overall  preparation  for  getting 
a  civilian  job,”  fairness  of  performance  evaluations,  and  special  pay.  Married  personnel  were 
more  satisfied  than  single  personnel  across  categories,  although  having  children  did  not  greatly 
affect  the  satisfaction  of  single  or  married  personnel  on  most  issues. 

ACTS  satisfaction  items:  Highest  and  lowest  reason  for  leaving  percentages.  The  ACTS 
responses  revealed  a  clear  pattern  in  “reason  for  leaving”  ratings.  Among  the  highest  percentage 
items  that  ACTS  respondents  perceived  as  major  reasons  for  leaving  were:  Amount  of  Time 
Separated  from  Family  (30.2%),  Respect  the  Army  Shows  for  Soldiers  (29.4%), 
Promotion/advancement  Opportunities  (25.3%),  Quality  of  Senior  NCO  Leadership  (24.4%), 
Overall  Quality  of  Army  Life  (23.6%),  Amount  of  Time  for  Family  and  Friends  (22.4%), 
Amount  of  Respect  from  Superiors  (22.3%),  Respect  Army  Shows  for  Single  Soldiers  (22.3%), 
Support  and  Concern  the  Army  Has  for  Your  Family  (22.2%),  Superiors’  Recognition  of  My 
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Accomplishments  (21.8%),  Level  of  Competence  of  Supervisors  (21 .7%),  Quality  of  Officer 
Leadership  (21 .2%),  and  Amount  of  Control  over  My  Job  Assignments  (20.1%). 

Among  the  lowest  percentage  items  that  ACTS  respondents  perceived  as  major  reasons 
for  leaving  were:  Army  Community  Service  Programs  (2.8%),  Army  Youth  Service  Programs 
(2.9%),  Overall  Assistance  Provided  by  the  Army  Career  and  Alumni  Program  (3.2%), 
Explanation  of  Transition  Benefits  and  Entitlements  (4.1%),  Morale,  Welfare,  and  Recreation 
(MWR)  Programs  (4.1%),  and  DoD  Dependent  Schools  (DODDS)  (4.4%). 

ACTS  satisfaction  items:  Important  demographic  differences  in  reason  for  leaving 
percentages.  Unlike  the  many  demographic  differences  in  satisfaction  ratings,  the  demographic 
differences  in  “Reason  for  Leaving”  percentages  were  comparatively  few.  In  general,  these 
differences  showed  that  females  were  more  likely  than  males  and  blacks  more  likely  than  whites 
to  believe  that  a  particular  aspect  of  Army  life  was  a  more  significant  reason  for  leaving. 
Similarly,  enlisted  personnel  were  more  likely  than  officers  to  believe  that  particular  aspects  of 
Army  life  were  significant  reasons  for  leaving,  although  these  differences  applied  more  to 
Privates  through  Specialists/Corporals  than  to  Sergeants  through  Sergeant  Majors/Command 
Sergeant  Majors. 

Marital  status  was  an  important  factor  in  determining  whether  an  aspect  of  Army  life  was 
a  significant  reason  for  leaving  the  Army,  with  larger  percentages  of  married  than  single 
personnel  rating  particular  aspects  of  Army  life  as  reasons  to  leave  the  Army.  Involuntary 
separations  and  those  who  were  not  seeking  a  similar  position  outside  the  Army  were  more 
likely  to  find  a  particular  aspect  of  Army  life  to  be  a  major  reason  for  leaving  the  Army  than 
voluntary  separations  or  those  who  were  seeking  a  similar  job.  Having  dependents  and  the  level 
of  one’s  education  were  factors  unrelated  to  “reason  for  leaving”  ratings. 

Relationship  between  satisfaction  and  reason  for  leaving.  As  would  be  expected,  the 
ACTS  responses  revealed  an  inverse  relationship  between  “satisfaction”  and  “reasons  for 
leaving.”  That  is,  items  with  the  highest  levels  of  satisfaction,  were  items  with  the  lowest 
percentages  of  ratings  as  major  reasons  for  leaving.  However,  all  but  one  of  the  items  that  had 
the  lowest  satisfaction  ratings  were  on  the  list  of  those  items  with  the  highest  percentage  of 
personnel  who  perceived  them  as  major  reasons  for  leaving. 

The  conclusion  from  this  finding  is  the  following:  Army  retention  would  likely  not  be 
increased  by  improving  aspects  of  Army  life  where  the  highest  satisfaction  is  already  evident,  but 
retention  might  be  positively  influenced  by  improving  those  aspects  of  Army  life  that  are 
important  to  soldiers  and  where  their  satisfaction  is  low. 

If  we  were  to  use  a  simple  and  informal  method  of  determining  which  issues  are  most 
troublesome,  we  could  subtract  the  percent  of  respondents  who  thought  that  an  issue  was  a  major 
reason  for  leaving  from  the  percent  of  respondents  who  were  satisfied.  Such  an  analysis  would 
provide  an  index  of  how  closely  the  issue  should  be  watched,  with  items  of  lower  value  being  of 
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most  concern.  Table  1,  which  contains  aspects  of  Army  life  listed  in  descending  order  of 
satisfaction,  reveals  that  items  of  most  concern  overlap  significantly  with  those  items  that  were 
rated  as  major  reasons  for  leaving  the  Army. 


Assistance  From  ACAP  89.7 

ACS  Programs  89.6 

Expl  of  Transition  Bnfts  83.0 

Army  Youth  Srvc  Prgms  77.5 

MWR  Rec  Services  75.2 

DoDDS  73.0 

Stateside  Living  Conditions  71.8 

Educational  Benefits  69.0 

Job  Security  68.8 

Military  Dental  Care  Quality  67.6 

Number  of  PCS  Relocation  66.7 

Assignment  to  Ldrshp  Jobs  65. 1 

Family  Medical  Care  Availability  61.7 

Army  Child  Care  Prgms  61.6 

PCS  Move  Compensation  60.4 

Geographic  Location  of  Job  60.5 

Overseas  Duty  59.7 

Prep  For  Getting  Civ  Job  58.9 

Use  of  Skills/training  on  Job  58.5 

Family  Dental  Care  Availability  57.9 

Family  Medical  Care  Quality  57.4 

Family  Medical  Care  Availability  56.6 

Amt  of  Enjoyment  From  Job  56.2 

Govt  Housing  Availability  54.6 

Military  Medical  Care  Quality  53.8 

Job  Fulfillment/Challenge  53.3 

Amt  of  Follow-on  Trng  For  MOS/AOC  50.6 
Respect  Army  Shows  Dual-Military  49. 1 

Govt  Housing  Quality  49.1 


Spouse  Career/work  Opportunity  48.9 

Tech/Prof  Dev  in  Job  Assignments  46.7 

Special  Pay  46.0 

Amount  of  Regulation/discipline  45.9 

Fairness  of  Performance  Evaluations  45.0 
Co-workers  Competence  44.8 

Retirement  Benefits  44.7 

Respect  Army  Shows  Sngle  Parents  41.5 

Amount  of  Basic  Pay  40.8 

Ldrs’  Technical/tactical  Competence  39.0 

Overseas  Living  Conditions  38.5 

Respect  Army  Shows  Spouses  38.4 

Family  Adjustment  of  Being  Army  Fam  36.9 
Respect  from  Superiors  35. 1 

Supervisors  Competence  33.8 

Quality  of  Officer  Leadership  33.4 

Control  over  Job  Assignments  33.3 

Amt  of  Personal  Time  32.5 

Recognition  for  Accomplishments  31.9 

Amount  of  Personnel  to  Do  Work  31.2 

Overall  Quality  Army  Life  30.4 

Quality  of  Sr  NCO  Leadership  29.5 

Promtn/Advancement  Opportunity  29.2 

Amt  of  Time  for  Family  &  Friends  24.5 

Respect  Army  Shows  Sngle  Sldrs  19.4 

Army  Sup  &  Concern  for  Family  18.8 

Respect  Army  Shows  Soldiers  12.9 

Amt  of  Time  Separated  from  Family  4.9 


Table  1:  An  Informal  Index  of  Troublesome  Items 

Objective  2:  A  Comparison  of  ACTS  and  SSMP  Responses 

Although  the  ACTS  and  SSMP  have  unofficially  and  inconsistently  co-existed  together  for 
five  years,  the  equivalence  of  their  data  has  never  been  investigated.  A  comparison  of  the 
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two  is  important,  since  there  is  considerable  overlap  among  the  items.  As  a  management  tool, 
some  have  claimed  that  exit  surveys  and  interviews  can  play  a  major  role  in  reducing  an 
organization's  voluntary  turnover  rate,  while  others  contend  that  they  are  of  questionable  value 
(Garretson  &  Teel,  1 982).  Personnel  studies  show  that  employees  are  likely  to  be  more  open  in 
their  comments  when  reporting  on  a  confidential  basis  (Goodale,  1982),  but  such  confidentiality 
could  theoretically  be  achieved  with  the  SSMP.  What  would  make  the  ACTS  data  unique  and 
valuable  is  that  such  data  could  be  more  critical  due  to  collection  at  the  end  of  soldiers’  careers. 
This  timing  of  data  collection  might  lead  to  more  candid  responses  from  separating  personnel. 
However,  if  the  responses  to  the  items  on  both  surveys  are  similar,  the  ACTS  would  offer  no 
unique  advantage  over  the  SSMP. 

Using  ARI’s  standard  to  determine  similarities  between  items  on  the  two  surveys,  32 
items  were  selected  that  were  similarly  stated  on  both  surveys.  A  log-linear  analysis  was 
performed  to  determine  whether  the  responses  to  the  two  surveys  were  consistent  overall  and  if 
there  were  any  particular  differences  within  demographic  category  responses.  Appendix  A 
summarizes  the  results  of  this  analysis. 

As  the  table  in  Appendix  A  shows,  using  a  significance  value  of  p  <  .05,  the  results 
demonstrated  that  there  was  no  overall  statistical  main  effect  difference  between  ACTS  and 
SSMP  scores.  Even  from  the  vantage  of  specific  demographic  differences,  the  few  differences 
are  not  systematic.  This  leads  to  the  conclusion  that  the  differences  seen  are  probably  due  to 
statistical  error,  found  at  some  level  in  all  surveys.  Essentially,  the  ACTS  and  SSMP  data  do  not 
differ  statistically  in  any  component;  hence  they  provide  statistically  identical  data. 

Objective  3:  Recommendations 

As  a  result  of  the  analyses  performed,  we  recommend  that  the  Army  discontinue  the 
ACTS.  It  is  rare  when  an  analysis  reveals  results  that  are  this  unequivocal.  Because  the  matching 
ACTS  and  SSMP  items  do  not  differ  statistically,  there  is  no  basis  for  the  continuation  of  the 
ACTS. 


This  recommendation  is  made  both  within  the  specific  context  of  the  data,  as  well  as 
within  the  more  general  context  of  the  exit  survey  literature.  Because  the  ACTS  does  not 
provide  the  Army  with  the  main  advantage  of  exit  surveys,  which  produce  more  candid 
responses  that  would  have  resulted  in  differences  between  the  surveys,  even  exit  survey 
proponents  such  as  this  investigator  would  find  it  difficult  to  support  its  continuation.  Further 
administration  of  the  ACTS  is  a  waste  of  both  money  and  personnel  time. 

Because  the  recommendation  is  to  discontinue  the  ACTS,  further  detailed 
recommendations  about  its  continuation  might  appear  to  contradict  this  advice.  Thus  these 
additional  recommendations  can  be  found  in  Appendix  B.  However,  we  strongly  urge  that  these 
recommendations  not  be  enacted  and  that  the  primary  recommendation,  discontinuation  of  the 
ACTS,  instead  be  executed. 
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CONCLUSIONS 


The  ACTS  is  an  instrument  that  provides  considerable  information  for  the  Army.  Given 
the  lack  of  difference  in  quality  or  content  with  data  in  the  SSMP,  however,  it  is  difficult  to 
justify  its  continued  existence.  Its  termination  seems  warranted,  if  only  from  the  standpoint  of 
cost  effectiveness.  Items  that  are  not  in  the  SSMP  but  are  in  the  ACTS  could  be  considered  for 
inclusion  in  the  SSMP  and/or  other  surveys  if  and  when  the  decision  to  terminate  the  ACTS  is 
made. 
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Results  of  Log-Linear  Analysis:  Common  Items  by  Subgroups 


£  8 
o  e 
<  o 

cl  L 

^  TO 

w  w 

CO 


cotocococococococococococotocotocococococococococo 

ZZZZZZZZZZZZZZZZZZZZZZZZZ 


£  35 

O  TO 

< 

Q_  <2 

85 

CO  TO 
CO 


cococococococococococococococococococol^cococococo 

ZZZZZZZZZZZZZZZZZZZZZZZZZ 


§* 

o 

Hi 

£ 


(d 

c 

< 


5 

w  x 

\—  (U 

O  CO 
«: 

0- 


C 0 
CO 


5 

CO 

h~ 

o 

CL 


«  CO 
CO  co 


5 

(O 

}— 

o 

Q. 


CO  ra 

w  co: 


5 

CO 

h- 

o 

< 

CL 

CO 

CO 


cococococococococococococococococococococococococo 

zzzzzzzzzzzzzzzzzzzzzzzzz 


cococococococococococococococococococococococococo 

zzzzzzzzzzzzzzzzzzzzzzzzz 


cococococococococococococococococococococococococo 

zzzzzzzzzzzzzzzzzzzzzzzzz 


cococococococococococococococococococococococococo 

zzzzzzzzzzzzzzzzzzzzzzzzz 


O  £ 


CQ 

o 

”3 

LU 

X 


* 

oc 

o 

§ 

o 

Q 

o 

h- 


LU 

o 

z 

LU 
I — 
LU 
CL 

o 

o 

CO 

DC 

LU 

or 

o 


o 

o 


CO 

h- 

z 

LU 

z 

o 

CO 

CO 

< 

CQ 

o 

“3 

cc 

LU 

> 

o 

_J 

o 

QC 

z 

o 

o 


LU 

z 

Q_ 

o 

CO 

D 

" — 

z 

o 
5 

3 

o 

LU 

*  o 


< 

3 


3  O 


£ 

LU 

LU 

o 
z 
< 

i  LU 


LU 


CO 

1- 

z 

LU 

§p 

CO 
< 


CO 
DC 
QL  O 

si  w 

2  P  3 


x  O 


£  o 
<  a: 

LL  CL 


x  — 

^  CO  CO 

IN  CO  LU 

y  <  * 


LU 


CO 

2  V 

Q  H 

b  □ 

Q  >-  CD 

z 
o 
o 
o 


CD  >- 


<  2 
D-  LU 
_ I  2 
<  LU 

o  £ 

LU  h- 
CL  LU 
CO  DC 


3  > 
tl  Cl  — 1 
££  co  LU 
3  <  Q 
O  LU  ^ 
LU  CO  ft 
CO  CC  lZ 


CD  LU 

o  > 


O  CO 


^3 

SI 

a  < 
o  o 
z  z 

CO  CO 
3  3 

o  o 

I  X 

£  £ 
o  o 
o  o 


sis 

U  h  (V 

o  <  z? 

J  W  A 

IU  z  u 

o:  LU  -u 

£8£ 

O  LU  S 

w  o 

CD  ^  5 

^  co  5 

3  0^ 
Z  CL  2 


CNOl'intONCOCD^T- 


intDNcocnOT-MfO 

t-T-T-r-T-CMCMC\|(N 


24.  MILITARY  DENTAL  CARE  QUALITY 

25.  FAMILY  MEDICAL  CARE  QUALITY 


Results  of  Log-Linear  Analysis:  Common  Items  bv  Subgroups 


5 

co 

I— 

o 

< 

CL 

CO 

if) 


if)  if)  if)  if)  if)  if) 

z  z  z  z  z  ^ 


>\ 

03 

C 

< 


if) 

3 

5 

TO 

CO 

I— 

<0 

o 

"to 

< 

~ 

X 

TO 

if) 

,  . 

if) 

TO 

CO 

5 

CO 

O  if)  | 

$ 

X  co  I 

2  w 

co 

CO 


if)  if)  if)  if)  if)  if) 

z  z  z  z  z  z 


CO  CO  CO  CO  CO 

In  z  z  z  2  z 


5 

CO 

h- 

o 

$ 

CL 

:> 

CO 

CO 


if)  if)  if)  if)  if)  if) 

z  z  z  z  z  z 


£ 

to 

CO 

t5 

H 

o 

$ 

IE 

O 

X 

z 

* 

if) 

TO 

if) 

if) 

if)  if)  if)  if)  if)  if) 

z  z  z  z  z  z 


5 

CO 

H 

o 

< 

X 


if)  if)  if)  if)  if)  if) 

z  z  z  z  z  z 


CO 

CO 


£  § 


CO 

s 

$ 

< 

LU 

X 

5 


< 

a 

1X1 
DC 
< 

_  ° 

H  £ 

2  5 

2  a 

>-  > 


>-  ca 

5 


CO 

O  >- 

a  h 

O  X 
Q  < 
w  3 
CO  o 

o  =J 

§  £ 
O  LU 
CO  > 


<  <  < 


Z  LU 

LU  LL 

X  ^ 
LU  CC 

Q  < 


(DSCOOJOt- 
CN  CN  (N  CN  CO  CO 


A-2 


Results  of  Log-Linear  Analysis:  Common  Items  by  Subgrou 


C/)WWW(/3WW(OW(/)C/}(/)(/)(/3WC/)WWWW(/)WC/3WW 

zzzzzzzzzzzzzzzzzzzzzzzzz 


GOCOCOCOtOCOCOCOCOCOCOCOCOCOCOCOCOCOCOCOCOCOGOCOCO 

zzzzzzzzzzzzzzzzzzzzzzzzz 


5 

— f 

CO 

H 

O 

o 

O 

< 

< 

a 

CO 

CO 

03 

CO 

WCOWO)CO(/3C/)WC/)(/)C/)C/)C/3W(/)WCO(0(/)COCO(0(/3WW 

zzzzzzzzzzzzzzzzzzzzzzzzz 


wwwwwwwwwwwwwwwwwwwwwcowww 

zzzzzzzzzzzzzzzzzzzzzzzzz 


m 

i 

< 


25.  Fam  Med  Qua! 


Results  of  Log-Linear  Analysis:  Common  Items  by  Subgroups 


$ 

CO 

h- 

o 

< 

CL 


CO  to 
CO  Wl 


CO  CO  CO  CO  CO  CO 
Z  Z  Z  Z  Z  Z 


-  Q- 
5  O 
CO  O 
0. 


O 

$ 

Q_ 

CO 

CO 


CO  CO  CO  CO  CO  CO 

z  z  z  z  z  z 


5  ?JI 


CO  CO  CO  CO  CO  CO 

z  z  z  z  z  z 


$ 

CO 

l_ 

o 

CL 


CO 


CO! 


CO  CO  CO  CO  CO  CO 

z  z  z  z  z  z 


S  TO  I 


CO  col 
CO 


CO  CO  CO  CO  CO  CO 

z  z  z  z  z  z 


5 


Q_ 

CO 

CO 


col 


CO  CO  CO  CO  CO  CO 

z  z  z  z  z  z 


CO 

f— 

o 

< 


§=  TO 

^  wl 

CO 


CO  CO  CO  CO  CO  CO 

z  z  z  z  z  z 


E 

TO 


„  TO  ro  ^  ^ 

'?  a  <  £  s 

TO  TO  TO  O 

T3  c  C  o  TO 

TO  (D  TO  0  w  ~ 

^  Q  Q  TOO-1 

r  r  r  D  o  > 

E  E  E  o  JZ  E 

TO  TO  TO  cl  o  t 

ll  ll  ll  co  CO  < 

tDNCOOlOr-' 

(M  C\J  (N  04  CO  CO 


A-4 


APPENDIX  B 


ACTS-RELATED  RECOMMENDATIONS 

Broadly,  these  recommendations  are  intended  as  guidelines  for  the  Army  to  consider. 
However,  if  the  Army  continues  the  ACTS  despite  our  recommendation,  it  should  closely  follow 
those  procedural  recommendations  that  impact  the  validity,  reliability,  and  utility  of  the  data. 
These  procedural  recommendations  represent  potential  safeguards  against  future  difficulties  with 
ACTS  data.  Nothing  in  the  recommendations  that  follow  should  be  construed  as  support  for  the 
continued  administration  of  the  ACTS. 

Recommendation:  Investigate  the  Varied  Meanings  of  Low  Satisfaction  and  Major 
Reasons  for  Leaving  the  Army.  Although  ACTS  data  provides  some  consistent  and  significant 
indications  of  problematic  aspects  of  Army  life,  many  of  the  items  that  are  identified  are  vague 
and  require  further  investigation.  This  in  itself  is  not  a  problem,  since  the  ACTS  was  not 
intended  to  provide  detailed,  specific  data;  instead,  it  offers  more  global,  general  data.  However, 
when  items  are  low  in  satisfaction  ratings  and/or  high  in  reason  for  leaving  ratings,  this 
ambiguity  should  be  mitigated. 

It  is  strongly  recommended  that  the  Army  investigate  the  precise  and  varied  meanings  of 
some  of  the  more  problematic  aspects  of  Army  life  identified.  For  example,  what  are  the 
deficient  components  of  “respect”  that  ACTS  respondents  see  as  a  major  reason  for  leaving  the 
Army?  What  “qualities”  of  Senior  NCO  leadership  are  deficient?  The  meanings  of  some  of 
these  statements  are  undoubtedly  multifaceted. 

An  investigation  of  this  type  can  be  done  using  a  nominal  group  technique  or  focus 
group,  and  can  follow  procedures  similar  to  those  used  effectively  by  this  investigator  in 
previous  military  diagnostic  work  (see,  for  example,  Giacalone  &  Rosenfeld,  1990;  Giacalone  & 
Knouse,  1 994).  Such  an  investigation  can  uncover  the  varied  meanings  of  some  of  these  aspects 
of  Army  life  and  can  help  to  identify  those  changes  needed  to  improve  Army  life. 

Recommendation:  Indices  for  the  ACTS  and  Changes  in  Items.  At  the  present  time,  it  is 
difficult  to  recommend  any  particular  indices  for  the  ACTS,  given  that  the  instrument  is  still  in 
its  infancy.  However,  as  the  survey  and  its  usage  matures,  we  would  recommend  that  indices  be 
considered. 

Although  at  least  one  index  (Giacalone,  R.  A.,  Elig,  T.  W.,  Ginexi,  E.  M.,  &  Bright,  A.  J., 
1 995)  has  been  used  scientifically  with  the  ACTS,  we  would  suggest  that  future  consideration  of 
indices  avoid  the  factor  analytic  approach  to  indexing.  This  approach,  which  focuses  on 
identifying  and  aggregating  indices  based  on  underlying  variables  (e.g.  supervision,  benefits), 
does  not  capture  the  multi-dimensionality  of  the  variables  well.  Essentially,  by  indexing 
variables  in  this  way,  data  is  “lost”  and  potentially  troublesome  areas  are  ignored.  Instead,  it  is 
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recommended  that  indices  be  developed  that  focus  on  changes  in  satisfaction/reason  for  leaving 
within  each  demographic  category. 

Thus,  baselines  for  each  of  the  “satisfaction/reason  for  leaving”  variables  can  be 
established  overall,  as  well  as  for  each  demographic  variable,  and  serve  as  independent  indices. 
These  baselines  can  index  both  overall  deviations  in  new  responses  and  particular  demographic 
deviations  in  new  responses.  Together,  each  demographic  index  (race,  gender,  education  level) 
can  be  assessed  against  the  overall  index  to  identify  particular  demographic  deviations  from  the 
overall  responses  as  well  as  against  itself  to  determine  whether  changes  within  a  demographic 
category  are  consistent  or  inconsistent  with  overall  changes. 

Relatedly,  we  do  not  recommend  any  additions  or  deletions  for  the  ACTS  at  the  present 
time.  The  items  have  face  validity  and  are  understandable,  thus  there  is  no  overt  reason  to  delete 
any  items.  Additions  should  be  made  so  that  form  follows  function:  Is  there  a  specific  data  need 
that  can  be  met  by  adding  an  item?  Such  a  rule  of  thumb  might  also  be  applied  to  indices  that  are 
considered  in  the  future:  Does  the  index  meet  some  data  need?  As  it  stands,  the  ACTS  is  a 
lengthy  survey  but  one  which  is  predicated  on  the  data  needs  of  various  Commands.  It  is  only 
when  these  Commands  require  new  data,  or  fail  to  use  existing  ACTS  data,  that  changes  might 
be  considered. 

Recommendation:  Monitor  and  Access  Sponsor  Satisfaction  With  the  ACTS.  Given  the 
expense  cost  of  the  ACTS,  extensive  monitoring  should  occur  to  assess  ACTS  sponsor 
satisfaction.  This  recommendation  is  based  on  current  expectations  regarding  service  quality  in 
an  organizational  setting.  Service  quality  expectations  in  today’s  business  and  organizational 
world  (in  the  public  and  private  sector)  require  organizations  to  provide  sufficient  service  quality 
in  a  number  of  different  dimensions.  These  major  dimensions  involve  credibility,  access, 
communication,  understanding  the  sponsor,  reliability,  responsiveness,  competence,  and 
courtesy.  It  is  important  that  the  ACTS,  as  a  new  instrument  already  tainted  by  problems, 
address  these  dimensions  in  order  to  be  viable.  A  brief  discussion  of  each  dimension  follows. 

Credibility  refers  to  the  trustworthiness  and  believability  of  the  product  or  service  that  is 
being  provided.  From  this  vantage,  the  question  is  not  whether  the  ACTS  is  a  credible 
instrument,  but  do  the  sponsors  believe  in  its  utility?  Can  they  trust  the  data?  The  problems 
inherent  to  military  exit  instruments  in  general,  and  the  ACTS  in  particular,  make  the  credibility 
of  the  revised  version  suspect.  It  is  important  to  address  this  issue  quickly. 

Access  involves  facility  or  ease  of  contact  with  which  sponsors  can  utilize  data  from  the 
ACTS.  How  easy  is  it  to  get  the  data?  How  easy  is  it  to  talk  to  someone  when  there  is  a 
problem?  Is  it  convenient  for  the  sponsor?  These  are  all  essential  issues.  If  the  ACTS  is  to 
perform  to  its  potential,  it  must  be  accessible  to  those  who  need  the  data. 
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Communication  focuses  on  the  extent  to  which  ARI  is  listening  to  those  people  who  are 
interested  in  the  ACTS  data.  Can  ARI  respond  to  sponsors  in  a  clear  manner?  Can  they  use 
language  that  is  easy  to  understand  and  avoid  technical  jargon?  Even  if  ACTS  data  is  excellent 
in  quality,  improper  communication  will  cause  significant  handicaps. 

Understanding  the  sponsor  concerns  whether  ARI  is  making  the  effort  to  know  the 
sponsors  and  what  their  needs  are.  Is  there  a  recognition  of  those  needs?  Are  those  needs  clearly 
understood  and  clearly  monitored?  What  processes  have  been  put  into  motion  that  will  help  ARI 
to  do  so? 

The  discussion  of  reliability  goes  beyond  a  technical,  scientific  definition.  In  addition  to 
the  methodological  concern,  the  Army  must  be  interested  in  the  ability  to  perform  the  promised 
service  dependably  and  accurately.  Does  the  data  arrive  on  time?  Does  it  arrive  in  the  needed 
format?  This  is  an  extremely  important  issue  because  it  impacts  a  sponsor’s  ability  to  use  the 
data  that  they  receive. 

Responsiveness  refers  to  ARI’s  willingness  to  help  the  sponsor  and  provide  prompt 
service  when  there  is  a  difficulty.  For  example,  can  ARI  resolve  a  data  need  quickly  if  the 
sponsor  requests  it?  Is  ARI  willing  to  answer  questions  regarding  data  other  than  that  which  is 
reported  (e.g.,  specific  breakouts  that  were  not  available)?  To  date,  the  singular  format  for  ACTS 
reporting  limits  its  responsiveness. 

Competence  refers  to  the  perception  that  those  who  are  analyzing  and  administering  the 
ACTS  possess  the  skills  and  knowledge  required  to  perform  the  service  provided.  Perceptions 
translate  to  sponsor  action  and  use  of  the  data.  For  example,  if  the  sponsor  has  a  phone 
encounter  with  a  staff  member  who  is  fumbling  for  information,  does  not  seem  to  know  how  to 
access  the  data,  or  is  unfamiliar  with  the  specific  ways  that  the  data  is  presented  to  the  sponsor,  a 
perception  will  develop  that  there  are  incompetent  persons  administering  the  ACTS.  Though 
erroneous,  when  sponsors  begin  to  see  ACTS  from  this  vantage,  they  are  likely  to  be  less 
supportive,  less  comfortable  with  it,  and  less  likely  to  feel  it  can  be  relied  upon.  Perceived 
competence,  conversely,  translates  into  higher  utility  and  application  of  data. 

Courtesy  cannot  be  underplayed.  Polite,  respectful,  considerate,  and  friendly  staff,  even 
at  the  lowest  levels  of  contact,  are  extremely  important.  If  the  ACTS  is  to  succeed,  people  must 
feel  that  it  is  an  instrument  supported  with  a  friendly  staff .  One  of  the  problems  with  ACTS  in 
the  past  was  not  necessarily  a  lack  of  courtesy,  but  a  sense  that  there  was  disinterest  in  the 
ACTS,  leaving  no  one  who  could  provide  needed  data.  This  is  extremely  important  and  must  be 
corrected  in  the  future. 

Given  the  concern  with  service  quality,  what  should  the  Army  assess  regarding  the 
ACTS?  It  is  easy  to  talk  about  these  dimensions,  but  unless  specific  strategies  are  detailed,  it 
will  be  very  difficult  to  translate  them  into  practice.  The  literature  (see,  for  example,  Zeithaml, 
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Parasuraman,  &  Berry,  1 990)  indicates  that  a  number  of  items  must  be  closely  monitored  and 
assessed.  Among  these,  some  are  particularly  important: 

1 .  Level  of  Support  for  the  ACTS.  To  what  extent  is  there  support  by  sponsors 
for  the  ACTS  data? 

2.  The  Level  of  Perceived  Credibility  of  the  ACTS.  How  credible  is  the  ACTS  to 
the  sponsor?  Do  they  trust  the  data?  Do  they  see  it  as  something  important  in 
terms  of  their  day-to-day  needs  and  how  to  respond  to  those  needs? 

3.  Level  of  Long-Term  Commitment  to  the  ACTS.  Is  the  ACTS  being  used 
temporarily,  or  is  there  a  sense  of  long-term  commitment  to  monitor  changing 
trends  for  future  decisions? 

4.  Level  of  Importance  of  the  ACTS  Data  to  the  Units.  Commands/units  may  say 
that  they  would  like  to  have  the  data,  but  how  do  they  use  it?  How  important  is  it 
to  them?  Is  it  an  essential  part  of  their  data  needs  or  used  simply  because  it  is 
there? 

5.  Level  of  Usage.  This  goes  beyond  level  of  importance.  At  what  level  of 
magnitude  is  the  ACTS  used?  The  level  of  usage  can  involve  a  number  of  items 
used,  number  of  times  the  data  is  accessed,  or  number  of  issues  that  the  data 
impacts  once  it  is  used.  This  is  very  important  in  determining  whether  the  ACTS 
is  having  an  impact. 

6.  Historical  Level  of  Success.  It  is  important  that  the  Army  determine  whether 
the  ACTS  influenced  decisions  in  an  effective,  accurate  way.  Success  means  that 
the  ACTS  provided  data  which  the  sponsor  or  its  units  were  able  to 
use/implement.  Thus,  it  is  important  that  the  ACTS  data  be  monitored  in  terms  of 
success  level.  Has  the  data  helped  decision-making?  Has  the  data  provided 
crucial  information?  Has  the  data  prevented  problems  or  located  opportunities? 

These  are  very  important  issues.  Once  again,  if  the  Army  cannot  determine 
whether  the  ACTS  has  had  an  impact,  it  is  of  no  use  to  the  Army. 

Recommendation:  Assess  Level  of  Satisfaction  with  Servicing  the  ACTS.  People  with 
good  intentions  still  may  leave  their  sponsors  dissatisfied  with  the  service  that  they  provided. 
Thus,  to  monitor  the  level  of  sponsor  satisfaction  servicing  doesn’t  indicate  that  ACTS  servicing 
is  poor.  It  is  a  recognition  that  level  of  satisfaction  must  be  monitored  because  satisfaction  is 
often  dependent  on  specific  issues  that  staff  outside  the  command  may  not  recognize.  Close 
monitoring  of  servicing  satisfaction  will  help  to  be  responsive  to  sponsor  needs  and  provide 
them  with  appropriate  data. 
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In  this  appendix,  we  provide  two  methods  of  assessing  the  ACTS:  interview  and  survey. 
In  the  initial  ACTS-related  work  by  HumRRO,  interviews  of  the  various  transition  sites 
successfully  revealed  many  problems.  As  a  result,  the  interview  protocol  which  follows  these 
recommendations  attempts  to  similarly  gather  data  regarding  satisfaction  problems.  The 
interview  protocol  format  provides  a  good  way  of  gathering  detailed  information  and  is 
undoubtedly  the  appropriate  way  to  start  the  process.  Much  detailed  information  can  be  gathered 
in  this  way  in  order  to  accurately  assess  functioning,  meeting  of  sponsor  needs,  and  sponsors’ 
comments/  questions.  Although  this  is  costly,  it  is  helpful  at  the  early  stages  of  implementation. 

Following  the  interview  protocol  is  a  survey  protocol  which  could  be  implemented  after 
the  first  year  that  the  ACTS  is  fully  operational.  At  that  time,  many  of  the  problems  with  the 
ACTS  will  have  been  eliminated  and  much  of  the  data  needed  to  make  the  survey  functional  will 
have  been  gathered.  This  survey  protocol  would  be  a  less  expensive,  less  time-intensive  way  to 
gather  data.  This  would  not  mean  low  quality  data,  but  data  collected  specifically  for  a  database 
to  help  monitor  changing  needs,  expectations,  and  success  levels  for  the  ACTS. 

This  type  of  assessment  is  very  important  in  the  long  term  for  the  ACTS.  Literature  on 
exit  surveying  and  interviewing  (Giacalone  et  ah,  1 995),  as  well  as  data  collected  on  the  ACTS 
and  similar  military  instruments  used  by  the  Navy,  reveal  that  lack  of  success  has  been  largely 
the  result  of  a  lack  of  data  monitoring  and  misunderstanding  of  sponsors’  needs.  An  assessment 
process  should  be  implemented  immediately  along  with  a  decision  to  make  the  ACTS  fully 
operational. 

Recommendation:  Assess  and  Procure  New  ACTS  Sponsors.  The  ACTS,  in  its  present 
format,  is  an  expensive  instrument  that  collects  much  data.  However,  it  is  underutilized.  The 
Army  must  continue  to  procure  new  ACTS  sponsors.  An  interview  protocol  similar  to  the  one 
used  during  the  initial  creation  of  the  revised  version  would  provide  a  good  method.  Procuring 
new  sponsors  does  not  necessarily  mean  that  the  instrument  will  grow  in  size  (although  it  might). 
It  does  mean  that  ACTS  data  could  be  used  by  other  sponsors  to  help  inform  their  decisions, 
thereby  reducing  the  per  decision  cost  of  its  usage. 

It  is  strongly  recommended  that  a  list  of  the  commands  be  made,  that  ARI  systematically 
call  contact  people  in  each  of  these  commands,  and  engage  them  in  discussions  regarding  the 
ACTS  and  how  it  might  help  them.  In  the  long  term,  this  will  yield  new  sponsors  and  also  help 
to  strengthen  the  support  for  the  instrument. 

Recommendation:  Monitor  Administration  Sites.  Giacalone  (1993)  suggested  that  the 
administration  of  the  ACTS  was  inconsistent  and  problematic.  In  a  follow-up  study  with 
HumRRO,  these  concerns  were  found  to  be  accurate.  The  new  means  of  administration 
suggested  by  HumRRO  could  solve  some  of  these  problems,  but  some  monitoring  of  the  efficacy 
of  these  changes  must  occur.  It  is  recommended  that  the  Army  follow  an  established  interview 
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protocol  and  that  this  protocol  be  used  to  continually  monitor  administration  sites.  The  goal  is  a 
consistent  system  for  administering  the  ACTS  across  different  sites  which  provides  unbiased 
information  from  exiting  personnel.  More  importantly,  monitoring  whether  all  personnel  are 
provided  the  opportunity  to  complete  this  instrument  (or  whether  it  is  being  administered 
haphazardly,  as  was  the  case  in  the  previous  interviews)  will  help  to  determine  the 
representativeness  of  ACTS  respondents. 

Recommendation!  Provide  Automatic  Reports  to  Sponsors.  Assessment  of  satisfaction 
with  the  ACTS  helps  determine  whether  it  provides  data  in  the  format  sponsors  need.  We  would 
recommend  very  specific,  automated  formats  of  reports  to  each  of  the  sponsors.  All  sponsors 
should  not  be  given  the  same  format  data,  but  should  only  be  provided  with  the  data  requested. 
This  will  help  to  eliminate  confusion  and  difficulty  in  reading  reports.  Included  in  these  reports 
to  sponsors  should  be  an  indication  of  changes  in  responses  over  a  6-  to  12-month  period  on  the 
items  of  interest,  as  well  as  the  different  breakdowns,  such  as  race  and  gender,  that  they 
specifically  request.  If  sponsors  need  additional  data  or  complete  data  analysis  outputs,  these 
should  be  provided  to  them  as  well. 

Recommendation:  Maintain  An  On-Going  Database  and  Consistent  Analyses.  In  order 
to  use  the  ACTS  data  effectively,  it  is  important  to  use  a  consistent  set  of  analyses  from  an 
established  ACTS  database.  The  primary  benefit  of  such  a  database  is  to  provide  longitudinal 
data  that  show  trends  over  time.  The  current  system  of  briefing  and  reporting  quarterly  responses 
is  ineffective  and  problematic  because  quarterly  responses  represent  variations  in  the  data  that  are 
usually  insubstantial.  The  use  of  a  database  provides  an  effective  means  of  monitoring  ACTS 
responses  longitudinally.  Temporary  variations  in  responses  should  not  seem  as  an  indicant  of 
specific  problems,  but  as  another  data  point  to  be  monitored,  perhaps  more  carefully. 

The  ACTS  data  will  not  provide  any  reliable  information  if  the  system  of  quarterly  reports  is 
used.  The  Army  needs  to  carefully  look  at  items  which  deviate  at  any  particular  time  and  to 
maintain  a  closer  monitoring  of  those  items.  Certainly,  a  precipitous  spike  or  dip  in  an  item 
warrants  immediate  attention,  not  necessarily  based  on  that  information  alone,  but  on  a 
confluence  of  data  which  indicates  that  the  data  is  accurate  and  supportive  of  other  information. 


Similarly,  it  is  important  to  perform  the  same  analyses  and  use  the  same  format  to  present 
the  data.  For  example,  currently  the  Army  uses  frequencies  of  responses.  Once  the  decision  is 
made  to  make  the  ACTS  fully  operational,  it  will  be  essential  to  decide  how  the  data  will  be 
analyzed  and  presented.  If  the  decision  is  to  use  frequency  data,  then  it  is  important  to  continue 
to  look  at  frequency  data  and  not  switch  to  means,  factor  scores,  or  other  types  of  data.  Although 
changing  the  type  and  presentation  of  analysis  is  certainly  possible,  a  database  must  be 
maintained  so  that  any  analytical  technique  can  be  analyzed  and  presented  longitudinally. 

Recommendation:  Computerize  the  ACTS.  The  suggestion  to  computerize  the  ACTS 
has  been  made  a  number  of  times,  most  recently  by  HumRRO  in  1995.  The  technological 
advantage  of  the  branch  capability  has  already  been  described  previously,  so  it  will  not  be 
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detailed  in  this  report.  However,  in  light  of  the  rapid  development  of  computer  technology,  the 
computerization  of  the  ACTS  is  even  more  important  and  necessary.  The  new  possibility  of 
using  a  Web  site  to  administer  ACTS  offers  a  real  advantage  that  allows  for  exit  surveying 
without  the  costs  of  individual  software  and  disks  that  would  otherwise  be  needed. 

From  Computer  Assessment  to  Web  Assessment.  The  use  of  a  Web  site  provides  many 
significant  advantages.  First,  the  data  can  be  used  quickly  because  it  is  directly  entered  into  a 
database.  Second,  the  Web  site  provides  easy  access  for  personnel  who  are  exiting  by  using  one 
or  more  computer  terminals:  there  is  no  need  to  transfer  information  to  floppy  disks,  record 
data,  or  experience  delay  in  the  period  from  response  to  analysis.  Third,  because  Web  graphics 
create  an  interesting,  interactive  format,  exiting  personnel  should  be  more  excited  by  the 
process.  The  enormous  success  of  the  Web  indicates  a  high  level  of  interest  and  should 
generalize  to  the  ACTS.  Fourth,  Web  sites  would  increase  the  effectiveness  of  the  ACTS  by 
facilitating  modifications,  thereby  potentially  speeding  new  changes  to  the  next  exiting  group  of 
personnel.  This  is  a  major  advantage  over  traditional  computer  administration  (using  a  DOS  or 
Windows  format)  which  restricts  changes  to  diskettes  and  requires  that  disks  be  sent  to  the 
remote  sites.  A  Web  site  allows  direct  and  immediate  changes  and  eliminates  the  long  time  lag 
between  change  and  implementation.  Certainly,  if  quick  response  is  needed,  this  saves  at  least 
two  to  three  days,  and  eliminates  having  to  deal  with  remote  units  to  explain  and  implement 
changes. 

Finally,  Web  administration  does  not  mitigate  the  effectiveness  of  “paper  and  pencil”  or 
computer  ACTS  administration;  it  enhances  all  previous  methods.  A  pilot  Web  administration 
would  be  relatively  simple,  since  conversion  of  the  current  ACTS  format  to  basic  HTML  can  be 
done  with  WordPerfect  or  Word.  Such  a  pilot  program  would  give  the  Army  a  way  to  assess  the 
effectiveness  of  Web  administration,  identify  glitches,  evaluate  the  time  needed  to  complete  the 
ACTS,  and  determine  the  optimal  number  of  computer  units  needed.  The  comparative  interest 
level  between  “paper  and  pencil”  and  Web  site  administration,  as  well  as  the  quality  of  the  data 
that  results,  could  also  be  assessed  at  that  time. 

Recommendation:  Calculate  the  ACTS  Return  on  Investment  (ROI).  A  fundamental 
issue  is  whether  the  ACTS  is  cost  effective  for  the  Army.  Essentially  this  is  a  question  of  return 
on  investment  (ROI).  How  does  one  calculate  ROI  for  the  ACTS?  Specialists  in  training  and 
development  have  used  a  methodology  to  calculate  ROI  for  training  and  development  that  can  be 
extrapolated  and  modified  for  use  in  determining  whether  the  ACTS  ROI  is  adequate  (Phillips, 
1996).  This  is  a  financially  based  approach  to  calculating  whether  it  is  useful  beyond  scientific 
utility.1  In  Stage  1,  the  Army  needs  to  identify  the  potential  utility  of  the  ACTS  data,  determined 
by  finding  out  from  various  units  and  commands  how  they  plan  to  use  the  data.  In  Stage  2,  the 
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1  Financial  benefits  are  but  one  aspect  of  ROI.  We  can  also  measure  other  “returns”  on 
investment:  What  is  the  increase  in  morale?  Preparedness?  These  are  different  “returns”  on  the 
investment  which  do  not  focus  particularly  on  financial  benefits. 


Army  determines  the  general  extent  to  which  the  data  has  been  utilized  in  the  intended  way.  Has 
the  planned  data  utility  been  realized?  Often,  with  instruments  such  as  the  ACTS,  the  instrument 
is  planned  for  a  particular  use,  but  is  later  found  to  be  useful  in  other  ways.  Because  the  data 
could  be  used  in  ways  that  had  not  originally  been  planned,  in  Stage  2A  the  Army  should 
ascertain  whether  the  data  was  utilized  in  other  (unplanned)  ways  (e.g.,  used  data  to  improve 
retention).  Once  these  determinations  have  been  made,  in  Stages  3  and  3A,  the  Army  identifies 
the  specific  ways  that  the  ACTS  data  has  been  utilized  or  implemented  (e.g.,  used  data  to 
improve  retention  in  sub-unit  X).  In  Stage  4  of  both  planned  utilization  and  new  utilization,  the 
Army  can  assess  measurable  results  that  have  been  produced.  This  requires  calculating  how 
ACTS  data  has  been  used  and  what  implementation  results  have  achieved.  For  example,  the 
Army  might  ask  what  would  have  happened  if  the  unit/commands  had  not  possessed  this  data; 
alternatively,  the  Army  may  wish  to  compare  baseline  information  (prior  to  using  ACTS  data)  to 
post-data  results  (after  using  ACTS  data). 

Finally,  in  Stage  5  and  6,  a  dollar  assessment  is  made.  Stage  5  is  particularly  important, 
for  what  was  learned  from  Stage  4  is  converted  into  dollar  equivalents  for  each  of  the 
commands/units  that  have  requested  the  data.  Here,  the  savings  resulting  from  the  multiple  uses 
of  the  ACTS  data  are  aggregated.  How  much  total  money  was  saved?  To  what  extent  did  the 
Army  profit  directly  as  a  result  of  the  ACTS  data?  In  Stage  6,  the  separate  units/commands 
calculations  from  Stage  5  are  added  together.  This  total  dollar  value  represents  the  savings 
gained  by  using  the  ACTS.  This  total  dollar  payoff  is  then  divided  by  the  total  overall  cost  of  the 
ACTS.  The  total  ratio  should  exceed  1  for  a  good  investment;  the  higher  the  ratio,  the  greater  the 
ROI. 


Ultimately,  an  ROI  calculation  allows  the  Army  to  go  beyond  scientific  assessment  and 
functionally  assess  the  degree  of  long-term  “payoff’  for  the  Army.  This  calculation,  of  course, 
must  be  ongoing  because  ROI  can  change  over  time,  making  it  a  less  effective  human  resource 
investment  for  the  Army.  This  appendix  also  contains  a  chart  showing  a  variety  of  methods  that 
can  help  the  Army  determine  how  to  calculate  ROI  for  the  ACTS. 


B-8 


References  for  Appendix  B 


Giacalone  (1993).  A  Critical  Evaluation  of  the  Army  Career  Transitions  Survey:  A  Suggested 
Approach  for  Reformation  (SSP-TCN  93-3001  Triangle  Park,  NC:  Battelle  Research, 
Inc. 

Giacalone,  R.  A.,  Elig,  T.  W.,  Ginexi,  E.  M.,  &  Bright,  A.  J.  (1995).  The  impact  of  identification 
and  type  of  separation  on  measures  of  satisfaction  and  missing  data  in  the  exit  survey 
process.  Military  Psychology,  7, 235-252. 

Giacalone,  R.  A.  &  Knouse,  S.  B.  (1989).  Farewell  to  fruitless  exit  interviews.  Personnel,  66, 
60-62. 

Giacalone,  R.  A.  &  Rosenfeld,  P.  (1990).  Family  separation  and  petty  regulations  as  dissatisfiers 
on  the  Navy  separation  questionnaires.  (NPRDCTN-90-13).  San  Diego:  Navy 
Personnel  Research  and  Development  Center. 

Phillips,  J  .J.  (1996,  April).  How  much  is  the  training  worth?  Training  and  Development,  20-24. 

Zeithaml,  V.  A.,  Parasuraman,  A.,  &  Berry,  L.  L.  (1990).  Delivering  quality  service:  Balancing 
customer  perceptions  and  expectations.  New  York:  Free  Press. 


B-9 


INTERVIEW  PROTOCOL 


1 .  Please  describe  how  you  have  used  ACTS  data  in  the  past  year. 

2.  Please  describe  how  you  initially  planned  to  use  the  data. 

3.  Does  the  data  serve  your  needs? 

4.  Does  the  data  provide  you  with  comprehensive  information?  If  not,  what  information 
could  the  ACTS  include  to  better  meet  your  needs? 

5.  In  the  past  year,  how  have  you  used  ACTS  data  to  improve  your  unit,  identify  /resolve 
problems,  or  identify/utilize  opportunities? 

6.  In  the  past  year,  how  has  ACTS  data  been  ineffective  in  serving  your  needs? 

7.  What  specific  changes  in  the  past  year  have  been  the  direct  result  of  ACTS  data? 

8.  Can  you  give  an  estimate  of  how  much  time  and/or  money  the  ACTS  data  might  have 
saved  you?  What  is  the  basis  of  that  estimate?  Are  there  any  performance  records  (e.g. 
accident  rates,  downtime,  efficiency,  production  schedules,  etc.)  that  might  support  your 
estimate? 

9.  Does  the  ACTS  provide  you  with  the  right  type  and  amount  of  data  to  meet  your  needs? 
If  not,  why? 

10.  How  might  the  ACTS  be  changed  to  better  meet  your  data  needs? 

1 1 .  Are  you  aware  of  any  other  data  sources  that  would  be  as  effective  as  the  ACTS? 

12.  Can  you  identify  other  units  that  might  be  interested  in  using  the  ACTS?  If  so,  please 
identify  the  unit  and  a  contact  person,  if  possible. 

13.  Are  you  confident  with  the  accuracy/credibility  of  the  data  the  ACTS  provides? 

14.  Are  you  satisfied  with  the  comprehensiveness  of  ACTS  reports/briefings? 

15.  Are  you  satisfied  with  the  timeliness  of  the  ACTS  reports/briefings? 

16.  Are  you  satisfied  with  the  clarity  of  the  ACTS-related  reports/briefings? _ 
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1 7.  Are  you  satisfied  with  the  organization  of  ACTS  reports/briefings? 

1 8.  Are  you  satisfied  with  the  helpfulness  of  ARI  staff  in  answering  ACTS-related  questions? 

1 9.  Overall,  is  there  anything  that  you  would  like  to  tell  us  about  ACTS  data? 

20.  How  important  is  ACTS  data  to  your  unit? 

21 .  Do  you  see  any  change  in  your  data  needs  that  could  be  met  by  revising,  or  including 
new  questions  on  the  ACTS? 

22.  For  which  of  the  following  reasons  are  you  using  the  ACTS?  (Check  all  that  apply) 

_ diagnosis  of  problem  areas 

_ ongoing  monitoring 

_ assessment  of  changing  needs 

_ tool  for  planning  or  forecasting 

_ for  use  in  recruitment  data 

_ data  to  help  in  personnel  development 

Figure  B-l.  Interview  Protocol 
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SATISFACTION  SURVEY 


PARTI 

Directions.  For  each  of  the  statements  below,  use  the  following  scale: 

1 .  Not  at  all 

2.  Very  little 

3.  Uncertain 

4.  Somewhat 

5.  Extremely 

1 .  To  what  extent  does  ACTS  data  serve  your  needs? 

2.  To  what  extent  does  the  data  provide  you  with  comprehensive  information?  If  not,  what 
information  could  the  ACTS  include  to  better  meet  your  needs? 

3.  In  the  past  year,  to  what  extent  have  you  used  ACTS  data  to  improve  your  unit, 
identify/resolve  problems,  or  identify/utilize  opportunities? 

4.  In  the  past  year,  to  what  extent  has  ACTS  data  been  effective  in  serving  your  needs? 

5.  To  what  extent  does  the  ACTS  provide  you  with  the  right  type  and  amount  of  data  to 
meet  your  needs? 

6.  To  what  extent  are  you  confident  with  the  accuracy/credibility  of  the  data  the  ACTS 
provides? 

7.  To  what  extent  are  you  satisfied  with  the  comprehensiveness  of  the  ACTS 
reports/briefings? 

8.  To  what  extent  are  you  satisfied  with  the  timeliness  of  the  ACTS  reports/briefings? 

9.  To  what  extent  are  you  satisfied  with  the  clarity  of  the  ACTS  reports/briefings? 

10.  To  what  extent  are  you  satisfied  with  the  organization  of  the  ACTS  reports/briefings? 

11.  To  what  extent  are  you  satisfied  with  the  helpfulness  of  ARI  staff  in  answering  ACTS- 

related  questions? 

12.  To  what  extent  is  the  ACTS  data  important  to  your  unit? 
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13.  For  each  the  following,  evaluate  the  extent  to  which  you  are  using  the  ACTS  data  for  this 
purpose? 

_ diagnosis  of  problem  areas 

_ ongoing  monitoring 

_ assessment  of  changing  needs 

_ tool  for  planning  or  forecasting 

_ for  use  in  recruitment  data 

_ data  to  help  in  personnel  development 

PART  2 

Directions.  Please  provide  additional  information  regarding  the  ACTS.  Give  as  much  detailed 
information  possible. 

1 .  Briefly  describe  how  you  have  used  ACTS  data  in  the  past  year. 

2.  Briefly  describe  how  you  initially  planned  use  ACTS  data. 

3.  What  specific  changes  in  the  past  year  have  been  the  direct  result  of  ACTS  data? 

4.  Can  you  give  an  estimate  of  how  much  time  and/or  money  the  ACTS  data  might  have 
saved  you?  What  is  the  basis  of  that  estimate?  Are  there  any  performance  records  (e.g. 
accident  rates,  downtime,  efficiency,  production  schedules,  etc.)  that  might  support  your 
estimate? 

5.  How  might  the  ACTS  be  changed  so  as  to  better  meet  your  data  needs? 

6.  Are  there  other  data  sources  that  would  be  as  effective  as  the  ACTS? 

7.  Can  you  identify  other  units  that  might  be  interested  in  using  the  ACTS?  If  so,  please 
identify  the  unit  and  a  contact  person,  if  possible. 

8.  Overall,  is  there  anything  that  you  would  like  to  tell  us  about  ACTS  data? 

9.  Do  you  see  any  change  in  your  data  needs  that  could  be  met  by  revising  (or  including 
new  questions)  on  the  ACTS? 


Figure  B-2.  Satisfaction  Survey 
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METHODS  TO  CALCULATE  ROI  LOR  THE  ACTS 


METHOD 

EXPLANATION 

EXAMPLE  OF  A  SPECIFIC  ACTS 
APPLICATION 

Converting  General  Data 

Usage  to  Contribution 

Conversion  to  corresponding 
dollar  equivalents  those 
direct  improvements  resulting 
from  use  of  ACTS  data 

Improvements  resulting  from  responses 
to  the  ACTS  item  "Amount  of  personnel 
available  to  do  work,"  can  be  converted 
to  dollar  savings  based  on  improvements 
from  shifting  personnel  resources  to 
needed  areas,  shifting  training  focus  to 
needed  work  areas,  etc. 

Calculating  Cost  of  Quality 

Conversion  to  corresponding 
dollar  equivalents  for  those 
quality  improvements 
resulting  from  use  of  ACTS 
data 

Improvements  resulting  from  responses 
to  the  ACTS  item  "Quality  of  family 
medical  care,"  can  be  converted  to  dollar 
savings  based  on  improvements  to 
medical  care's  impact  on  absenteeism, 
illness,  tasks  completed,  downtime,  etc. 

Converting  Personnel's  Time 

Conversion  to  corresponding 
dollar  equivalents  for  the 
amount  of  personnel  time 
saved  resulting  from 
improvements  due  to  ACTS 
data 

Improvements  resulting  from  responses 
to  the  ACTS  item  "Use  of  my  skills  and 
training  on  the  job,"  can  be  converted  to 
dollar  savings  based  on  improvements  to 
appropriate  skill  use,  improved  morale, 
and  appropriate  training  for  Army  needs 

Using  Historic  Costs 

Using  historical  baseline 
costs,  conversion  to 
corresponding  dollar 
equivalents  based  on 
improvements  from  a 
historical  baseline  made  as  a 
result  of  ACTS  data 

Improvements  resulting  from  responses 
to  the  ACTS  item,  "Quality  of  family 
dental  care,"  can  be  converted  to  dollar 
savings  based  on  improvements  to 
morale,  family  satisfaction  with  Army 
life,  etc. 

Using  Internal/Extemal 

Experts 

Using  costing  data  from 
intemal/extemal 
experts,  the  conversion  of  an 
expert's  estimates  of 
improvements  made  as  a 
result  of  ACTS  data  to  a 
corresponding  dollar 
equivalent 

Improvements  resulting  from  responses 
to  the  ACTS  item,  "Overall  job 
fulfillment/challenge,"  can  be  converted 
to  dollar  savings  based  on  improvements 
to  morale,  re-enlistments,  positive  public 
relations,  etc. 
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Using  External  Studies 

Using  costing  data  from 
external  studies,  the 
calculation  of  dollar  savings 
based  on  improvements  made 
as  a  result  of  ACTS  data 

Improvements  resulting  from  responses 
to  the  ACTS  item,  "Amount  of 
educational  benefits,"  can  be  converted 
to  dollar  savings  based  on  improvements 
to  benefit  plans,  implementation  of  new 
programs,  and  extension  of  successful 
education  benefits 

Using  Leadership, 

Supervisory  and  Personnel 
Estimates 

The  calculation  of  dollar 
equivalents  using  leadership, 
supervisory,  and  personnel 
estimates  of  improvement 
made  as  a  result  of  ACTS 
data 

Improvements  resulting  from  responses 
to  the  ACTS  item,  "Overall  quality  of 
Army  life,"  can  be  converted  to  dollar 
savings  based  on  improvements  to 
aspects  of  Army  life  which  result  in 
higher  morale,  satisfaction,  and 
improved  performance 

Using  Command  Estimates 

The  calculation  of  dollar 
savings  using  estimates  from 
the  Commands  regarding 
known  improvement,  based 
on  changes  made  as  a  result 
of  ACTS  data 

Improvements  resulting  from  responses 
to  the  ACTS  item,  "Army  child  car 
programs,"  can  be  converted  to  dollar 
savings  based  on  improvements  to 
morale,  lower  family  stress,  lower 
family  illness  and  injuries,  and  higher 
family  satisfaction  with  Army  life 

Table  B-l.  Methods  to  Calculate  ROI  for  the  ACTS 


